Depends on #88009, #88078

The Problem

ESCN was a discrete scene format that was created by the Godot team for ease-of-export from Blender to Godot. The plugin was originally written for Godot 3.x and used its scene format, and has not since been updated.

However, while Godot still has support for importing this format, ESCN importing has been broken since Godot 4.0 was released due significant changes in the scene format that conversion was not implemented for:

• Skeletons had a Transform pose property which was relative to bone rest in 3.x , which was broken out to Vector3 pose_position; Quaternion pose_rotation; Vector3 pose_scale that are absolute in 4.x.
• Animations used to have a transform track type with transforms relative to bone rest in 3.x, which was similarly broken out into position_3d, rotation_3d, and scale types that are absolute in 4.x
• The shaders that were included in the ESCN export were of the 3.x format, which is not compatible with 4.x.

Solution

This adds the following features to the ESCN editor importer:

1. Converts 3.x Skeletons to use the new pose properties and recomputes them to be absolute rather than relative to the bone rest.
2. Converts 3.x Animations with transform tracks into position, rotation, and scale tracks that are absolute rather than relative to the bone rest.
3. Converts old shader code to 4.x-compliant shader code

Design Considerations

The reason this is a draft right now is because of the following issues that I need advice on. This will likely be broken up into seperate PRs based on the feedback.

ResourceLoader modification to allow special handlers for a single load

I had issues with converting Animations and Shaders; basically, we can't handle converting them in either of the _set() functions for these classes like we can for Skeleton3D because:

• Animation tracks are stored ini style:

  

  This means that, upon load, each of those indexed properties gets set, one at a time. We need to split the transformation track up into position, rotation, and scale tracks, and we can't arbitrarily add tracks in the middle of a load, since the index of the tracks will change.

• Shaders compile their code upon setting the code property; if the code fails to compile, code won't be set. Additionally, it will result in a very, very long error message that will obliterate the error logs and significantly confuse the user.

• Both of those class names are the same from 3.x to 4.x, so I couldn't implement another class for either of those to handle the loading either.

So, to solve these issues, I modified the ResourceLoader to allow the addition of special handlers. Basically, we can add a handler for a specified Resource class that takes in a MissingResource and returns a Resource. If we try to load a resource, and the resource loader detects that it has a handler for this resource class, it will instead load the resource as a MissingResource and, after the resource and its properties are finished loading, it will hand it off to the handler to instantiate the resource and set its properties.

I had considered modifying the ResourceLoader classes to just add the property to missing_resource_properties if it fails to be set and then just getting those properties out of the metadata, but as mentioned above, this would result in a bunch of error messages (1000+ lines in total length) every time we tried to import an ESCN, which is not ideal.

The way I have it now, I'm not really sure if this is the correct design; this sort of special handling needs to be only for a single load rather than a global handler, so I decided to only add it to the instantiated ResourceLoader classes rather than the static ResourceFormatLoader classes. This breaks the pattern a bit, as it looks like it's intended that the user doesn't use the instantiated ResourceLoader class and instead uses the ResourceFormatLoader class. I'd appreciate some advice on how to implement this such that it can be used for a single load and doesn't break the pattern here.

ShaderLanguage token stream emitting

Converting Shaders from 3.x to 4.x is a bit more complicated than how it's currently handled in the 3.x to 4.x converter. In order to do a proper conversion of a 3.x to 4.x, we need to do the following:

• Replace everything in RenamesMap3To4::shaders_renames
• SCREEN_TEXTURE, DEPTH_TEXTURE, and NORMAL_ROUGHNESS_TEXTURE were removed, so their usage necessitates adding a uniform declaration at the top of the file
• If shader_type is "particles", we need to rename the function void vertex() to void process()
• CLEARCOAT_GLOSS was changed to CLEARCOAT_ROUGHNESS and its usage was inverted (i.e. ascending values now increase the roughness rather than decreasing it). So, we need to invert all usages of CLEARCOAT_GLOSS:
  • invert all lefthand assignments:
    • CLEARCOAT_GLOSS = 5.0 / foo;
      • becomes: CLEARCOAT_ROUGHNESS = (1.0 - (5.0 / foo));,
    • CLEARCOAT_GLOSS *= 1.1;
      • becomes CLEARCOAT_ROUGHNESS = (1.0 - ((1.0 - CLEARCOAT_ROUGHNESS) * 1.1));
  • invert all righthand usages
    • foo = CLEARCOAT_GLOSS;
      • becomes: foo = (1.0 - CLEARCOAT_ROUGHNESS);
• async_visible and async_hidden were removed, so these render modes need to be removed
• specular_blinn and specular_phong render modes were removed, and we can't handle these, so we just throw an error
• MODULATE is not supported (yet) in 4.x, so we have to throw an error.

As you can see, this is a bit more complicated than a simple regex search and replace would allow for, in particular the CLEARCOAT_GLOSS replacement. We at least need a token stream in order to properly detect for these conditions and modify the code.

So, in order to do this, I modified ShaderLanguage to emit a token stream:

• The function token_debug_stream() was added to emit a parsed token stream for the entire file.
• Token was modified to add position and length members so that we can emit code based on the token stream; this is neccesary primarily because the token parser can modify both identifiers and constant values before creating the token.
• The tokens TK_TAB, TK_CR ,TK_SPACE, TK_NEWLINE, TK_BLOCK_COMMENT, TK_LINE_COMMENT, and TK_PREPROC_DIRECTIVE were added and _get_token() was modified to emit them, but only if we are doing a debug parse. The reason for adding these was so that we can easily emit the new code from the token stream without obliterating the original formatting and comments

With these modifications, we're able to easily detect for the above conditions and insert and remove tokens from the token stream, and then emit code based on that token stream.

I'm a bit more confident about how I modified ShaderLanguage here but I'd still like to get feedback on how I modified ShaderLanguage here, and if there are any other 3.x to 4.x differences that I need to handle that I missed.